Repository navigation
DEV-1408: Keep the callback consumer thread alive when a handler throws - #344
Merged
Merged
Conversation
ConsumerThread caught only InterruptedException, so a RuntimeException from processMsgFromCallback() ended the thread for good. Its queue then stopped draining, producers blocked in put() once it held 1024 messages, and anything that later sent a callback hung - including an application's shutdown hook, which left the JVM running with no window. Each message is now processed in processMsgSafely(), which logs a RuntimeException and lets the thread carry on. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
BasicProcessWithCallBack.ConsumerThreadcaught onlyInterruptedException. ARuntimeExceptionthrown by anyprocessMsgFromCallback()implementation ended the consumer thread for good, and nothing reported it (underjavawthe stack trace goes nowhere). After that:LinkedBlockingDeque, capacity 1024) stopped draining;put(), and so did every producer upstream of those;This was found in a Consensys process that had stayed alive like this for five days after its window was closed. A thread dump showed two queues full at exactly 1024 messages, with no consumer left on the top-level one.
Fix
The consumer now handles each message through
processMsgSafely(), which catches aRuntimeException, logs the thread name, message identifier and stack trace toSystem.err, and carries on with the next message.Errors are still not caught.Tests
BasicProcessWithCallBackTest: a handler throws on the first message, and the test checks that the later messages are still processed and the thread is still alive. It fails on the unpatched class ("message after the exception was never processed") and passes with the fix.ShimmerDrivergradlew test: 178 tests, 0 failures.Reaching consumers
This needs a driver release and pin bumps before ShimmerAndroidAPI and ASM_PC (submodule) pick it up. Consensys builds the driver from source at its release tag.
Other host APIs: the queue/consumer-thread pattern exists only in this Java driver. The C#, Python, TypeScript, Swift and Rust implementations have no equivalent, so nothing to port.
A companion change in the Consensys API bounds its shutdown hook with a timeout, so that a stuck cleanup can no longer keep the process alive.
🤖 Generated with Claude Code